RHQ Agent Communications Services

When The RHQ Agent starts for the first time, it enters setup mode. You can also manually enter this setup mode by using the --setup command line option or use the setup agent prompt command. Once in setup mode, you will be prompted to provide values for a series of preference settings. Some of these settings involve setting up the communications services of the agent. Other settings are only prompted if you enter "advanced" setup mode or "all" setup mode (these "advanced" and "all" settings are marked with a (a) below). To enter advanced setup mode and thus be able to set advanced settings, use the --advanced command line option with --setup or use the prompt command setup advanced. To enter the "all" setup mode (which allows you to set every preference available), you must use the prompt command setup all.

• Agent Hostname or IP Address - this is the address that the agent will bind to when listening for incoming messages. This usually is the same address that the agent's remote clients (aka RHQ Servers) will use when trying to connect to the agent. However, in some network setups, this may not always be the case (e.g. a remote client going through a router that forwards requests to a different host). If you want the RHQ Server to connect to this RHQ Agent via a different address, you need to set up some special Transport Parameters to indicate this, as described below.

• Agent Port - this is the port that the agent will actually be listening to. This usually is the same port that the agent's remote clients (RHQ Servers) will use when trying to send messages to the agent. However, in some network setups, this may not always be the case (e.g. a remote client going through a router that forwards requests to a different port). If you want the RHQ Server to connect to this RHQ Agent via a different port, you need to set up some special Transport Parameters to indicate this, as described below.

• Agent Transport Protocol(a) - this is the transport the agent expects incoming messages to adhere to. This is usually "socket" or "sslsocket" (for raw binary socket messages, either unsecured or secured). The RHQ Agent does not host a servlet container, so you cannot use a servlet based transport (servlet, sslservlet) for server-to-agent communications. You can only use servlet based transports for agent-to-server communications (see the "RHQ Server Transport Protocol" preference below).

• Agent Transport Parameters(a) - these are additional parameters to be used when the agent creates its communications services and when its remote clients (RHQ Servers) needs to talk to the agent. See the section below, Transport Parameters, for the different types of transport parameters you can set.

• RHQ Server Hostname or IP Address - this is the IP address for the endpoint of the primary RHQ Server this agent will talk to. This RHQ Server IP Address value is dictated by the way the RHQ Server has configured its communications services (using similar settings as the ones being described for the agent). Please refer to your RHQ Server's communications configuration to see what exact value this should be.

• RHQ Server Port - this is the port that the primary RHQ Server is listening to. This RHQ Server Port value is dictated by the way the RHQ Server has configured its communications services (using similar settings such as these being described for the agent). Please refer to your RHQ Server's communications configuration for what exact value this should be.

• RHQ Server Transport Protocol(a) - this is the transport the primary RHQ Server will expect its incoming messages to flow over. This RHQ Server Transport value is dictated by the way the RHQ Server has configured its communications services (using similar settings such as these being described for the agent). Please refer to your RHQ Server's communications configuration for what exact value this should be.

• RHQ Server Transport Parameters(a) - this are additional Transport Parameters that are to be used when the agent connects to the primary RHQ Server. This RHQ Server Transport Parameters value is dictated by the way the RHQ Server has configured its communications services (using similar settings such as these being described for the agent). Please refer to your RHQ Server's communications configuration for what exact value this should be. In particular, you need to know what additional transport parameters the RHQ Server wants its clients to define. This is especially important if the RHQ Agent needs to connect to a different host and/or port than what the RHQ Server actually binds to.

• Command Send Timeout(a) - this is the amount of milliseconds the agent will wait before aborting a command (i.e. the amount of time in milliseconds that the RHQ Server has in order to process commands and return its results). Please ensure that this value is the same as the timeout specified in the transport parameters of your RHQ Server URI (if specified), since both timeouts will be enforced. If this Command Send Timeout value is less than or equal to 0, the agent will not timeout its messages (note that if a timeout is specified in the RHQ Server URI transport parameters, that timeout will be enforced).

• Command Send Retry Interval(a) - This is the minimum amount of time, in milliseconds, the agent will wait before trying to resend a guaranteed command that previously failed.

• Command Send Max Retries(a) - If a guaranteed delivery message is sent, but the agent fails to connect to the server and deliver the message, it will always be retried. However, if the error was something other than a 'cannot connect' error, the command will only be retried this amount of times before the command is dropped (at which time it will be considered lost forever).

• Maximum Commands To Concurrently Send(a) - This is the maximum number of commands the agent can send to the server at any one time. If you defined clientMaxPoolSize in your RHQ Server URI transport parameters, make sure its value is the same as this "Maximum Commands To Concurrently Send" value since you effectively cannot have one higher than the other.

RHQ Server Communications Services

Settings analogous to those described above exist on the RHQ Server side as well. In order to set these communications configuration preferences on the RHQ Server, you edit their values in the <server-install-dir>/bin/rhq-server.properties file. The RHQ Server must be restarted for any new values in the properties file to take effect.

Note that the rhq.communications.connector.transport-params setting allows you to set custom Transport Parameters just like you could do for the agent. Also note that these RHQ Server endpoint settings:

* rhq.communications.connector.transport
* rhq.communications.connector.bind-address
* rhq.communications.connector.bind-port
* rhq.communications.connector.transport-params

hold the values that your agent will need to know in order to talk to this server.

For example, if the following settings are used to configure your RHQ Server:

rhq.communications.connector.transport=socket
rhq.communications.connector.bind-address=192.168.0.10
rhq.communications.connector.bind-port=16163
rhq.communications.connector.transport-params=enableTcpNoDelay=true&backlog=200

then when you setup your RHQ Agent, its configuration values would be:

RHQ Server Hostname or IP Address=192.168.0.10
RHQ Server Port=16163
RHQ Server Transport Protocol=socket
RHQ Server Transport Parameters=enableTcpNoDelay=true&backlog=200

Transport Parameters

Both the RHQ Server and RHQ Agent use JBoss/Remoting as its underlying remoting framework. JBoss/Remoting defines remote endpoints (which identify how to connect to a RHQ Server or RHQ Agent) via InvokerLocator strings, which look like simple URLs. An example is socket://myhost.corp.com:16163/?transportParam1=value1&param2=value2. InvokerLocators consist of a transport protocol (socket: ) as well as the host and port of the remote endpoint. Also as part of an InvokerLocator, are transport parameters which further customize the endpoint. They help define the behavior of the underlying communications connector (which is the thing that accepts incoming messages from remote clients). The RHQ Server and RHQ Agent can each define their own transport parameters via their rhq.communications.connector.transport-params preference setting (the agent's advanced setup mode will prompt you with Agent Transport Parameters when asking for this value). Transport parameters are appended to the end of the InvokerLocator - in the same way a query string is appended to a URL. When you define your transport parameters, you must set them using the same syntax, specifically, multiple transport parameters are separated by ampersand characters. If you set transport parameters inside your XML config file, be sure you use the proper & string to represent the ampersand. Below you will find listed all the possible transport parameters that can be defined in your RHQ Server or RHQ Agent rhq.communications.connector.transport-params setting. They are split into two groups: the first group are settings that affect the server-side socket behavior and the second group are settings that affect the client-side behavior. In this context, "server-side" refers to the connector that accepts incoming messages and "client-side" refers to the remote clients that send outgoing messages. RHQ Servers and RHQ Agents have both a server-side and client-side since they both send and receive messages from each other.

• Settings that affect the server-side

  • serverBindAddress - The address on which the server socket binds to listen for requests. The default is an empty value which indicates the server socket should be bound to the host provided by the InvokerLocator URL (the host). This would be needed in the case that the client will be going through a router that forwards requests made externally to a different IP address or hostname internally (e.g. NAT setup).

  • serverBindPort - The port to listen for requests on. This would be needed in the case that the client will be going through a router that forwards requests made externally to a different port internally.

  • timeout - The socket timeout value. The default on the server side is 60000 (one minute). If the timeout parameter is set, its value will also be passed to the client-side (see below).

  • backlog - The preferred number of unaccepted incoming connections allowed at a given time. The actual number may be greater than the specified backlog. When the queue is full, further connection requests are rejected. Must be a positive value greater than 0. If the value passed if equal or less than 0, then the default value will be assumed. The default value is 200.

  • numAcceptThreads - The number of threads that exist for accepting client connections. The default is 1.

  • maxPoolSize - The number of server threads for processing client requests. The default is 300.

  • socket.check_connection - indicates if the invoker should try to check the connection before re-using it by sending a single byte ping from the client to the server and then back from the server. This configuration needs to be set on both the client and server to work. The default value is false.

• Settings that affect the client-side

  • timeout - The socket timeout value. The default on the client side is 1800000 (or 30 minutes).

  • enableTcpNoDelay - can be either true or false and will indicate if the client socket should have TCP_NODELAY turned on or off. TCP_NODELAY is for a specific purpose; to disable the Nagle buffering algorithm. It should only be set for applications that send frequent small bursts of information without getting an immediate response. The default is false.

  • clientMaxPoolSize - the client-side maximum number of active socket connections. This basically equates to the maximum number of concurrent client calls that can be made from the socket client invoker. The default is 50.

  • numberOfRetries - number of retries to get a socket from the pool. This basically equates to the number of seconds the client will wait to get a client socket connection from the pool before timing out. If the max retries is reached, a CannotConnectException will be thrown. The default is 30.

  • numberOfCallRetries - number of retries for making the invocation. This is unrelated to numberOfRetries in that when this comes into play is after it has already received a client socket connection from the pool. However, it is possible that the socket connection timed out while waiting within the pool. Since a connection check is not done by default, the connection is thrown away and an attempt to get a new one will be made. This will happen for however many numberOfCallRetries is (which defaults to 3). However, when (numberOfCallsRetries - 2) is reached, the entire connection pool is flushed under the assumption that all connections in the pool have timed out and are invalid and will start over by creating a new connection. If this still fails, a MarshalException is thrown.

  • socket.check_connection - Indicates if the invoker should try to check the connection before re-using it by sending a single byte ping from the client to the server and then back from the server. This configuration needs to be set on both client and server to work. This if false by default.

The example below illustrates how a single InvokerLocator URL with transport parameters can configure both server-side and client-side behavior. Let's assume we are configuring a RHQ Agent with the following settings:

• transport: socket

• bind-address: 192.168.0.5

• bind-port: 56565

• transport-params: backlog=300&enableTcpNoDelay=false

The full RHQ Agent URI would then look like: socket://192.168.0.5:56565/?backlog=300&enableTcpNoDelay=false. The backlog transport parameter is a server-side configuration and the enableTcpNoDelay transport parameter is a client-side configuration. When the RHQ Agent creates its server socket and begins listening for incoming messages from RHQ Servers, it sets its backlog to 300. The RHQ Agent ignores the enableTcpNoDelay parameter because it is only useful for clients that want to talk to its server socket. When a RHQ Server sends a message to this RHQ Agent, the RHQ Server will ignore the backlog parameter because it is strictly a server-side setting but it will disable its TCP_NODELAY setting. As you can see, a single InvokerLocator can provide useful information for both ends of the communications channel (client and server).

rhq.communications.connector.transport=socket
rhq.communications.connector.bind-address=public
rhq.communications.connector.bind-port=22222
rhq.communications.connector.transport-params=serverBindAddress=local&serverBindPort=11111